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ABSTRACT 

To determine the effects of the television program, 
"Around the Bend" and the Related activities of the home visitor and 
mobile van teachers, the Appalachia Preschool Test (APT) was 
developed by the Appalachian Educational Laboratory. APT is used to 
supplement the standardized instruments being used to measure program 
performance. The test consists of four subtests and an experimental 
section: Part 1 deals with color naming, identifying body parts, and 
right-left discrimination; Part 2 (described in this report) is the 
basis for determining specific cognitive learning from the early 
childhood education curriculum; Parts 3 and 4 are Piaget-type, 
conservation of number and size tasks; Par 5 is an experimental 
subtest designed to measure understanding of cause and effect, 
logical classification and letter recognition. To measure the 
achievement of cognitive objectives Part 2 was given to 273 children 
in three treatment groups, and subsequently to a 60-child sample in a 
comparison group. Children who viewed only the television program and 
were not exposed to the home visitor and mobile van teachers scored 
significantly lower on the test. Results indicate that the home 
visitor* more than any other part of the program, has a great 
potential for influencing the child's behavior, especially if she can 
produce changes in the child's environment. A summary of AEL Early 
Childhood Education program is available as PS 004 889. (Author/AJ) 
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ATTAINMENT OF COGNITIVE OBJECTIVES* 

Introduction 

The Appalachia Preschool Test was developed by AEL to supplement the 
standardized instruments which were being used to measure program perfor- 
mance. It consists of four subtests and an experimental section which was 
being considered for final inclusion in the test. 

Part 1 of the APT comprises 16 questions dealing with color naming, 
identifying body parts, and right-left discrimination. It was not derived 
from specific program objectives and tended to be more of an introduction 
to the remainder of the test. 

Part 2 of the instrument is described in the text of the following 
report, and is the basis for determining specific cognitive learning from 
the ECE curriculum. 

Parts 3 and 4 of the APT are Piaget-type, conservation of number 
and size tasks which together comprise 15 items. 

Part 5 is an experimental subtest which was designed to measure 
understanding of cause and effect, logical classification, and letter 
recognition. It was not included for the purposes of summative evaluation 

Part 2 of the APT is the only direct measure of cognitive tasks which 
have been included in the ECE curriculum, and for this reason it is the 
only subtest which is included for analysis in this section on the attain- 
ment of cognitive objectives. 



*This report was written by Mr. Brainard Hines of the Research and 
Evaluation Division. 
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METHOD 

During the program year which began in September , 1969 , and extended 
until June, 1970, a number of cognitive objectives from the Hooper-Marshall 
curriculum 1 were included in the primary objectives of the television pro- 
gram, Around the Bend , and in the related activities of the- home visitor 
and van teacher. 

To measure the achievement of these objectives, Part 2 of the Appa- 
lachia Preschool Test (APT) was administered in June, 1970, to a sample 
of 273 children in three treatment groups, and subsequently to a 60-child 
sample in a comparison group located in Morgantown, West Virginia. A more 
detailed description of sampling techniques is presented in the introduction 
to this report. 

Part 2 of the APT is a 61-item test, developed by the AEL staff in 
early 1969, and administered in June of that year to a sample of approxi-. 
mately 125 children in order to measure cognitive development as of that 
time. It consists of a variety of tasks, with the following number of 
questions composing the total: vocabulary-15 questions; relational terms- 

14 questions; letter and number recognition-6 questions; mathematical sets- 
4 questions; geometric shape-4 questions; and beginning and ending sounds- 
8 questions, with the balance of the test devoted to body parts, calendar 
dates, and time related terms. *' 

Each of the questions in Part 2 of the APT was derived from a 
corresponding objective which had been taught during the first year's pro- 
gramming (1968-69). The proportion of the various parts of the test to the 

^FrankH. Hooper and William H. Marshall, The Initial Phase of a Curriculum 
Development Project - Final Report (West Virginia University: Morgantown), 
August, 1968. 
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total instrument corresponds to the rank-ordering of the objectives taught 
throughout the year to the entire curriculum for that year. 

It was hoped that the curriculum of the second year's programming would 
closely resemble that of the first: year and that the APT could be used as 
a criterion referenced test for the second program year with no major re- 
visions. Examination of the programs used during the second year, however, 
indicates that the instrument does not adequately measure the program's 
objectives and has now become instead a measure of general knowledge and 
reasoning ability. 



ANALYSIS 



Table 3-1 lists mean scores and standard deviations for each sex and 
cell within the three treatment groups and one comparison group. 

These scores are combined for ages three, four, and five and both 
sexes in Figure 3-1, with means for each group represented graphically. 

Inspection of these scores leads to several possible conclusions. 

First, any instance where the control group scored higher than the TV only 
group is very probably due to sampling or measurement artifacts rather than 
treatment effects. Although a negative treatment effect is theoretically 
possible, the likelihood of the television program's reducing program- related 
learning is minimal. Since this pattern is evident for the APT, the 
assumption is made that differences existed in other significant dimensions 
outside of viewing Around the Bend . Second, if the APT did measure specific 
learning, the effects of the ECE program were not additive. That is, the 



television, home visitor, and mobile facility did not contribute equally to 
the specific cognitive learning. For example, an increased APT score was 
not observed with the treatment group which had the mobile facility in 
addition to home visitors and the television program. Third, the ranking 
of treatments according to increased mean scores on the APT (TV-HV, pack- 
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age, control, and TV only) indicates that the program treatments were not 
additive. That is, the mobile facility and paraprofessionals did not add 
constant amounts of cognitive learning to the basic level which was associ- 
ated with the television program. 



TABLE 3-1 



APPALACHIA PRESCHOOL TEST (PART 2) MEAN SCORES, 
STANDARD DEVIATIONS, AND SAMPLE SIZES BY AGE 
AND SEX WITHIN TREATMENT GROUPS 



x = 25.75 
SD » 5.12 

N « 8 


x = 26.00 
SD - 7.04 

N = 9 


x = 22.69 
SD = 6.30 

N = 13 


x = 24.15 
SD = 5.97 

N - 13 


x = 25.12 
SD = 7.51 

N = 8 


x = 29.10 
SD -• 9.55 
N = 10 


x = 20.00 
SD - 4.62 

N = 10 


x = 26.46 
SD = 5.24 

N = 13 


x = 33.23 
SD = 8.03 

N = 13 


x = 36.25 
SD = 8.26 
N = 8 


x = 22.88 
SD = 5.19 

N = 8 


x = 27.56 
SD = 6.75 
N = 13 


x - 32.27 
SD = 7.89 

N = 11 


x = 32.00 
SD = 7.36 

N = 10 


x = 28.08 
SD * 5.45 

N = 13 


x = 33.30 
SD = 8.45 

N = 10 



Pkg. TV-HV TV Only Comparison 
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A three-way analysis of variance procedure indicated (P<.0005) that 
a significant treatment effect existed. However , this inference resulted 
from the lower score for the TV only group, rather than a sequence of higher 
scores for all the treatment groups when compared with the control. A 
Scheffe post hoc comparison revealed that the TV only group scored significantly 
lower than the other two treatments but did not score significantly lower than 
the control group. The ANOVA table is reproduced in Table 3-2. An eta 
squared column is reported in the ANOVA table below indicating the percent 
of variance accounted for by each source in the table. 



TABLE 3-2 

ANALYSIS OF VARIANCE TABLE FOR THE APT, PART 2 



Source 


M 2 


df 


Mean Square 


F 


P 


I (trt) 


.118 


3 


404.5361100 


5.50 


.0005 


J (sex) 


.004 


1 


38.39546158 


0.8 




K(age) 


.031 


1 


1340.437420 


28.18 


< .0005 


IJ-INT 


.015 


3 


51.35871108 


1.08 




IK-INT 


.006 


3 


20.55948195 


0.43 




JK-INT 


.001 


1 


8.328900431 


0.18 




IJKINT 

Error 


.030 


3 


101.7390063 


0.18 






o 





The significant age effect shown is probably due to the achievement 
test format of the instrument. That is, if the items are relevant to the 
child's fund of general knowledge and reasoning ability, we would expect 
large increases with the passage of time. This is the case for the 1970 
ECE sample across all treatment groups. 
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A correlation matrix between Part 2 of the APT and the other instru- 
ments in the test battery is shown below. It provides further evidence for 
the general nature of the test. 



TABLE 3-3 

PEARSON r CORRELATIONS BETWEEN APT PART 2 AND OTHER TESTS 



Age 

in Months 


Frostig 
Total Score 


PPVT 

Raw Score 


PPVT 

M.A. 


PPVT 

IQ 


I TP A 
Total 


APT 1 


.45 


.68 


.63 


.63 


.50 


.73 


.60 



N = 150 (approx.) 



Although it is possible that the factorial impurity of the APT is 
responsible for this series of relatively high in ter correlations, it is 
more likely that the test is measuring a factor common to the »entire 
battery (see Technical Report No. 6 ) rather than the specific cognitive 
learning for which it was designed. 

Analysis of covariance (ANCOVA) routine, using chronological age 
and PPVT raw score as covariates, was performed on the results of Part 2 
of the APT. The ANCOVA provided no new information beyond that already 
obtained from the analysis of variance. 

: : . - ■ ; , . • " ' J ■ \ . 
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INTERPRETATION OF RESULTS 

It is difficult to make any inferences about the effectiveness of 
the program in teaching specific cognitive objectives from the results of 
the APT. Although some lessons from the first year's programming have been 
included in the second years' efforts, few of these programs' objectives 
correspond to questions on the APT. 

For this reason it does not seem proper to use the APT as the major 
criterion for program effectiveness. Success of the program was arbitrarily 
defined as achievement by children with IQ's of 90 and above of 90 percent 
of the objectives taught throughout the program year.* Computation of a 
"success" estimate for the second year would produce a figure that was 
spuriously low, since we wo'ald be measuring only a small portion of what 
was taught durii.j the year. An effort is currently being made to revise 
the APT to include all the cognitive objectives which have been taught 
throughout the entire three-year course of program development. When that 
instrument is available, we will be able to make a valid and reliable judg- 
ment of the program's success in teaching a set of cognitive objectives, 
and cf the true effects of each :.of the program components in enhancing that 
learning. 

At present, a few major conclusions can be drawn from the available 
data. The effects of the television program, paraprofessional , and mobile 
facility, insofar as they are reflected in the APT, are not equally impor- 
tant in bringing about the achievement of specific cognitive learning. 

2 The Laboratory, Evaluation Report : Early Childhood Education Pro- 

2 ram, 1969 Field Test (Appalachia Educational Laboratory: Charleston, 

West Virginia) , March, 1970, Appendix F, p. 14. 




This is not surprising since the home visito:* more than any other part of 
the program has a great potential for influencing the child's behavior. 

Any changes which she can produce in the home environment will be magnified 
in their effects on the child, since they will be more permanent and impor- 
tant in his life than exposure to the television program or mobile facility. 

The children who view Around the Bend and are not exposed to any of the 
other components do score significantly lower than those who receive the 
other treatments. The TV only group, although located geographically near 
the other two treatments, contains a higher percentage of rural families 
who are consequently somewhat deprived in their home environment. An 
alternative, and less likely hypothesis, is that the television program 
produces a negative treatment effect and thus reduces the scores of all three 
groups which receive that program component. • No support exists in the current 
data for this contention. 



